Objective: To assist the evaluation of central estimates produced by modellers, following the full model runs in Oct 2019, according to the VIMC minimum standards.
Checks:| 2 | cases for cholera-no-vaccination-test is greater than for cholera-campaign-default-test (see Table 7) |
| 4 | cases for cholera-no-vaccination-test is greater than for cholera-campaign-default-test (see Table 7) |
| 12 | deaths for cholera-no-vaccination-test is greater than for cholera-campaign-default-test (see Table 7) |
| 11 | deaths for cholera-no-vaccination-test is greater than for cholera-campaign-default-test (see Table 7) |
| 13 | dalys for cholera-no-vaccination-test is greater than for cholera-campaign-default-test (see Table 7) |
| 111 | dalys for cholera-no-vaccination-test is greater than for cholera-campaign-default-test (see Table 7) |
Specific characteristics of this model are shown in Table 1.
| details | |
|---|---|
| disease_name | Cholera |
| modelling_group | JHU-Lee |
| model_name | JHU-Lee-Cholera |
| model_type | static |
| Vaccine_1 | Cholera (SIA) |
| Vaccine_2 | NA |
| Vaccine_3 | NA |
| burden_min_age | 0 |
| burden_max_age | 100 |
| cohort_first | 1900 |
| cohort_last | 2100 |
| year_min | 2000 |
| year_max | 2100 |
| gender_specific | No |
| gender | Both |
| number_countries | 5 |
| number_expected_outcomes | 4 |
| 1 | cases |
| 2 | dalys |
| 3 | deaths |
| 4 | cohort size |
| number_expected_scenarios | 2 |
| 21 | Campaign, Default |
The modellers have been given 2 vaccination scenarios for this disease (Cholera). Each scenario contains one or more coverage sets. The characteristics of the scenarios are explained in the table below.
| scenario | coverage_set | vaccine | activity_type |
|---|---|---|---|
| Campaign, Default | Cholera: Cholera, with, campaign; test | Cholera | campaign |
| No vaccination | No vaccination | NA | NA |
The trends of the specific coverage sets can be observed in Fig 1. Notice that for campaign scenarios the coverage can be more than 100% in some cases for the interpolated population for that country/year/age combination.
Age ranges of the different coverage sets include either standard age ranges (as it is the case of Measles) but sometimes it differs by year and country so that several combinations can be possible (See table below). This is key to calculate the target population of a specific vaccine for an age/year/country combination.
| coverage_set_name | age_from | age_to | freq |
|---|---|---|---|
| Cholera: Cholera, with, campaign; test | 1 | 100 | 104 |
The estimates of fully vaccinated persons (FVPs) are obtained by multiplying coverage from each coverage set and the target population.
This calculation should reflect the different coverage sets included in each scenario. For details on the FVPs in all coverage scenarios by country, see Supplementary Figure 1.
The modellers have been given 3 outcomes for (Cholera). The characteristics of these outcomes are explained in the table below. Please notice that for some models the outcomes comprise several sub-outcomes that are reported in a separate form.
| outcome | definition |
|---|---|
| cases | Cholera cases |
| dalys | Cholera dalys |
| deaths | Cholera deaths |
This section is based on the uploaded scenarios where we also check that the estimates uploaded correspond to the expected outcome.
Table 5 shows the scenarios uploaded (specified in Table 2) and the specific touchstone for tracking purposes.
| scenario_description | touchstone | touchstone_name | uploaded_by |
|---|---|---|---|
| No vaccination | 201910gavi-5 | 201910gavi | elizabeth.lee |
| Campaign, Default | 201910gavi-5 | 201910gavi | elizabeth.lee |
Age and year ranges and the number of countries uploaded should correspond to the numbers expected given in Table 1.
| min_age_upload | 0 |
| max_age_upload | 100 |
| min_year_upload | 2000 |
| max_year_upload | 2100 |
| num_countries | 5 |
| scenario_description | outcome_code | number of rows |
|---|---|---|
| No vaccination | cases | 51005 |
| No vaccination | cohort_size | 51005 |
| No vaccination | dalys | 51005 |
| No vaccination | deaths | 51005 |
| Campaign, Default | cases | 51005 |
| Campaign, Default | cohort_size | 51005 |
| Campaign, Default | dalys | 51005 |
| Campaign, Default | deaths | 51005 |
The expected 'number of rows' in this table is 51005, (age classes x years x countries). If the number expected is different from the uploaded, please compare Table 1 against Table 3, and take a look at Fig. 1, 2 and 3 for some insight.
In Fig.3, we are comparing the observations expected for this model (blue line) against what has been uploaded by the modeller (orange area/dots). The missing/excess observations correspond to the area between the blue and orange lines. The figure aims to represent the birth cohort being followed over time, so that in a dynamic model we are expecting a rectangular shape whereas for a static model a 'trapezoidal' shape is the minimum expected. Some static models can also have a rectangular shape.
In Fig. 4 we are checking the patterns of zero values in the upload.
Zero values are normal in many cases, and we expect either absence, minimum presence or a random pattern of them. When a non-random pattern of zero values is observed, this plot can help in interpreting some of the intentional assumptions or systematic errors that cannot be seen in Fig. 3 (i.e. absence of estimates of an outcome in some age classes or in a whole birth cohort).
The cohort size is the number of people alive in a given birth cohort specified by the calendar year and age during that year, so it will be the same across all scenarios. We will then be able to calculate the number of FVPs (fully vaccinated persons) by multiplying this cohort size with the relevant coverage. The cohort size should be comparable to the interpolated population provided on Montagu. The cohort size should reflect the age range, time range and gender (female, male or both) for which the model is tracking the population.
In this section, the plots are specific to each model in terms of age range, year range, countries and gender. For each plot, the left panel reflects the demography figures provided in Montagu (interpolated population), the central panel represents the cohort size uploaded by modellers, and the right panel represents the difference between the two, given by cohort size - interpolated population. If the difference is positive, then the model has more cohorts than the interpolated population, and if negative, the model has fewer cohorts compared to the interpolated population.
The population used here from Montagu is type int_pop with source dds-201910 between 2000 and 2100.
Fig. 7 and 8 show absolute and relative difference respectively, between interpolated population and cohort size by country for this model.
In this section we are checking burden estimates for the different scenarios.
Here, we look at the aggregated (for all countries) age distribution patterns by outcome.
Figure 10 shows the total estimates of burden by decades between 2000-2030 (for the total number of countries) to see if the estimates are reasonable for the different scenarios.
Table 7 shows the total burden from all countries and age groups for no vaccination and vaccination scenarios. These values should correspond to the total across all ages from Fig. 11.
| outcome_code | scenario_description | number_of_people_millions |
|---|---|---|
| cases | No vaccination | 17.28 |
| cases | Campaign, Default | 16.65 |
| cohort_size | No vaccination | 54384 |
| cohort_size | Campaign, Default | 54384 |
| dalys | No vaccination | 25.94 |
| dalys | Campaign, Default | 25.13 |
| deaths | No vaccination | 0.6034 |
| deaths | Campaign, Default | 0.5808 |
Table 8 shows the total fvps for each scenario in the PINE countries for the same period as shown in Fig. 12. These values should correspond to the total across all years from Fig. 12.
| scenario_description | country | fvp_in_millions |
|---|---|---|
| cholera-campaign-default-test | ETH | 28.88 |
| cholera-campaign-default-test | IND | 1180 |
| cholera-campaign-default-test | NGA | 47.1 |
| country | scenario_description | burden_outcome | thousands |
|---|---|---|---|
| ETH | No vaccination | cases | 1321 |
| ETH | Campaign, Default | cases | 1306 |
| ETH | No vaccination | dalys | 842.6 |
| ETH | Campaign, Default | dalys | 834.3 |
| ETH | No vaccination | deaths | 16.48 |
| ETH | Campaign, Default | deaths | 16.32 |
The figure below shows burden for each outcome from the No vaccination scenario for all countries.
| ISO3 | Country | cholera-campaign-default-test | cholera-no-vaccination-test |
|---|---|---|---|
| COD | Congo, the Democratic Republic of the | 1 / 1 | 1 / 1 |
| ETH | Ethiopia | 1 / 1 | 1 / 1 |
| KEN | Kenya | 1 / 1 | 1 / 1 |
| SOM | Somalia | 1 / 1 | 1 / 1 |
| SSD | South Sudan | 1 / 1 | 1 / 1 |
| TOTAL | ALL COUNTRIES | 5 / 5 | 5 / 5 |